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ABSTRACT 

Synthetic Vision Systems (SVS) provide pilots with displays of stored geo-spatial data representing terrain, obstacles, and 
cultural features. As comprehensive validation is impractical, these databases typically have no quantifiable level of 
integrity. Further, updates to the databases may not be provided as changes occur. These issues limit the certification level 
and constrain the operational context of SVS for civil aviation. Previous work demonstrated the feasibility of using a real- 
time monitor to bound the integrity of Digital Elevation Models (DEMs) by using radar altimeter measurements during flight. 
This paper describes an extension of this concept to include X-band Weather Radar (WxR) measurements. This enables the 
monitor to detect additional classes of DEM errors and to reduce the exposure time associated with integrity threats. Feature 
extraction techniques are used along with a statistical assessment of similarity measures between the sensed and stored 
features that are detected. Recent flight-testing in the area around the Juneau, Alaska Airport (JNU) has resulted in a 
comprehensive set of sensor data that is being used to assess the feasibility of the proposed monitor technology. Initial 
results of this assessment are presented. 
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1. INTRODUCTION 

SVS provides pilots with a virtual visual depiction of the external environment. This depiction can be portrayed on any 
flight-qualified display media. Prototype implementations have used both head-down displays and head-up displays to 
provide pilots with aircraft state (e.g. altitude, attitude, airspeed), guidance and navigation information, and a perspective 
depiction of the terrain as viewed from the cockpit. Other types of information can also be presented such as obstacles, 
traffic, and weather. SVS has the potential to improve flight safety by increasing pilot situational awareness (SA) in low 
visibility conditions to a level of awareness similar to daytime clear- weather flying. NASA has been investigating SVS as a 
mitigation strategy for accident categories such as Controlled Flight Into Terrain (CFIT), runway incursions, low visibility 
and loss-of-control scenarios; along with the ability to fly advanced precision approach procedures. In addition, the Federal 
Aviation Administration (FA A) has mandated Terrain Awareness and Warning Systems (TAWS) for nearly all aircraft to 
increase pilot’s SA with respect to terrain. The major difference between TAWS and SVS is that SVS is being designed for 
applications ranging from purely advisory systems to flight-critical systems, whereas TAWS is currently required only as an 
advisory system. 

1.1 Integrity 

The term integrity is used frequently in the aviation community as a quality metric. Several segments of the community 
interpret integrity differently. There are three definitions of integrity that are relevant to the research presented in this paper: 
system integrity, data integrity, and data processing integrity. With respect to DEMs used by SVS, required data integrity 
will depend on the intended use of the data by the pilot and the architecture of the system in which the data resides. DEM 
integrity is related to system integrity in that system integrity can be compromised if errors exist in the DEM that may lead to 
Hazardous Misleading Information (HMI) being presented to pilots, and these errors are not detected by the operational 
system. 

To ensure that data is not corrupted during processing and/or distribution, the International Civil Aviation Organization 
(ICAO) has established guidelines for data processing integrity. ICAO defines data processing integrity as the degree of 
assurance that aeronautical data and its value have not been altered since the data origination or an authorized amendment 
[1]. To provide guidance to data processors and/or distributors, RTCA has published guidelines for data processing 
procedures that are intended to help ensure that the resulting data is no worse than the source data [2]. It is expected that the 



majority of terrain data that is stored on aircraft as part of an SVS or TAWS will not have a stated integrity with respect to 
the source data itself. The integrity specified with these data will only refer to data processing integrity. This is primarily 
due to the fact that the amount of validation required to establish an integrity value for such large data sets is viewed as cost 
prohibitive. 

1.2 Digital elevation models 

Geo-spatial terrain databases, that may be used to depict terrain information on SVS displays, are referred to as Digital 
Elevation Models (DEMs). A variety of sources from both the public and private sectors provide DEMs, and these DEMs are 
specified by a number of parameters. These parameters include the post-spacing or spatial resolution of elevation 
measurements, the horizontal and vertical references or datums, and the circular and linear error probabilities. The circular 
error probability (CEP) is used for the horizontal accuracy specification of the post location, whereas the linear error 
probability (LEP) is used to specify the accuracy in the vertical direction for the post elevation. Assuming that the circular 
and linear errors are random and normally distributed with zero-mean, the standard deviations in the horizontal and vertical 
directions, can be derived by dividing the 90% CEP and 90% LEP by 1.645. The standard deviation in the horizontal 
direction refers to the standard deviation of the error radius from the true post location using a two-dimensional polar 
coordinate frame. 

1.3 Integrity monitoring approach 

When utilizing DEMs in flight-critical systems, it is imperative to avoid the display of hazardous misleading terrain 
information (HMTI). HMTI can be the result of insufficient DEM spatial resolution, inappropriate coloring/texturing, or 
excessive DEM errors. The severity of the hazard will depend on the specific flight operation being conducted and the use of 
the terrain depiction by pilots during this operation. This paper describes an approach to real-time terrain database integrity 
monitoring that can reduce the probability of an undetected database error being inadvertently presented to the pilot. The 
proposed integrity monitor concept is an extension to the approach described in [3] [4] [5]. For the proposed monitor, sensor 
information from the Global Positioning System (GPS), an Inertial Reference Unit (IRU), and an X-band WxR are used in 
connection with a DEM to enable feature-based disparity checking. Disparities between features that are sensed and features 
that are extracted from the stored DEM are compared in a statistical manner similar to the one described in [5]. If significant 
inconsistencies exist, an integrity alert is generated. The remainder of this paper provides specifics on how this can be 
achieved by making use of a large data set that was recently acquired during flight-testing in the JNU region. 


2. X-BAND WEATHER RADAR OVERVIEW 

Previous work has shown how radar altimeters can be integrated with SVS to provide a level of integrity for a DEM 
[3] [4] [5]. Unfortunately, the radar altimeter approach has two shortcomings. First, because radar altimeters make 
measurements nominally from nadir, looking-down vertically, horizontal DEM errors are difficult to detect unless there is 
significant terrain undulation under the flight path. Second, unless the detected error is a bias or ramp-type error that persists 
over a large area of the DEM, the detection may not be operationally useful as the aircraft has already flown over the region 
where the error was detected. Both of these shortcomings led to the need to look at forward-looking sensors. 

In today’s aircraft fleet, by far the most common forward-looking sensor is the weather radar. Specifically, X-band pulse 
Doppler radars are used primarily to provide flight crews with a display of weather information and to provide forward- 
looking wind-shear detection capability. A secondary purpose, as described in [6], is “ground-mapping to facilitate 
navigation by display of significant land contours”. Although using the weather radar display to aid navigation was 
historically a pilot-specific talent derived from extensive use of the device, recent work has shown that integration with 
DEMs can be used to supplement on-board navigation systems and to detect potential ground-based hazards [7] [8] [9]. 

A typical certified WxR system includes a Receiver/Transmitter (R/T), an antenna, a radome, and a display. The WxR 
system is designed such that all signal processing is performed by the R/T and data products are provided to the display. As 
examples, the radome, antenna, and R/T for NASA’s B-757 are shown in Figure 1. The R/T installed on this aircraft is a 
Rockwell-Collins WxR-2 1 00. 

Details on the characteristics of commercial certified WxR systems can be found in [6] [10] [11]. Operating frequencies for 
X-band systems range from 9.36 to 9.56 GHz. Antenna diameters vary from 15 to 30 inches with gains ranging from 28.9 to 
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35 dB, respectively. Beam-width can vary from 2.9 to 6.0 degrees. Antennas scan up to +/- 90 degrees with respect to the 
longitudinal axis of the aircraft at a nominal rate of 15 looks per minute (45 degrees per second). Range can be set by the 
pilot to 5, 10, 20, 40, 80, 160, or 320 nmi; however, the number of range bins with reported reflectivity measurements 
remains fixed at 512. Each reflectivity measurement is quantized into four levels with a fifth used for severe windshear. For 
display, colors are associated with these levels for each range bin during scanning. Measurement interval time is nominally 6 
msec using a 0.25 degree azimuth resolution during scanning. The above specifications are taken from [6]. 



Figure 1. NASA B-757 radome, antenna, and WxR-2100 R/T 


3. FEATURE-BASED DISPARITY CHECKING 

Because the WxR provides measurements at a high rate, it is desirable to extract terrain- specific features from these 
measurements to reduce the complexity of the disparity checking (i.e. integrity monitoring) function. In addition, to quantify 
integrity and/or integrity bounds, a statistical approach is required. Because solving statistical problems involving 
combinations of several random variables can be intractable, a reduced, or minimum, set of features to use for disparity 
checking is desired. 

Figure 2 depicts the proposed algorithm for feature-based disparity checking using a forward-looking scanning sensor such as 
the WxR. Using this approach, there are two threads that operate in parallel: a sensor-derived thread and a DEM-derived 
thread. Ideally these threads would be completely independent to eliminate the possibility of a common-mode failure that 
may not be detectable. In Figure 2, the sensor-derived thread provides the sensor pointing direction (AZANG, TIFT) and the 
range setting to the DEM-derived thread. This inter-dependence, while not ideal, should not compromise integrity since 
failures of these data elements should be detectable (e.g. AZANG not changing during scanning). 

The first two stages of each thread are responsible for detecting features along each radial line measurement (i.e. within the 
512 range bins) then classifying these features. For the prototype version of the algorithm, shadow edge locations are used as 
the features of interest. Shadow edges are often the most significant feature discemable in WxR measurements and occur 
when reflectivity values transition to/from 0 dB indicating no detectable reflectivity. WxR shadowing occurs most frequently 
in areas of moderate to severe terrain undulations when the aircraft is at a relatively low altitude or the antenna depression 
angle (i.e. tilt) is large. Figure 3 depicts a segment of a sample WxR radial measurement and the edge features that would be 
detected and classified as either rising or falling edges by the prototype algorithm. The gradient at each edge can also be 
computed and used as a weighting factor to down-sample features prior to disparity checking. 

Feature detection and classification in the DEM-derived thread is similar; however, the challenge for this thread is to generate 
synthesized radial measurements. The prototype algorithm uses aircraft position from GPS, aircraft attitude from an IRU, 
antenna pointing direction from the WxR, a beam model, and a DEM to generate the synthesized measurements. Each 
measurement is created by finding the DEM height at each range bin location (1-512) along the azimuth pointing direction of 
the antenna, then finding the depression angle from these points on the DEM back to the WxR antenna location. These 
depression angles are then compared to the expected range of angles for the main beam for a particular measurement. The 
expected range of angles is tilt +/- one-half of the beam-width. Each range bin for a synthesized measurement is assigned a 
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value as follows: -3, denotes the DEM-derived depression angle is outside the range of the main beam; 1, denotes the angle 
is within the range of the main beam and therefore, energy should be reflected back to the WxR; 2, denotes an edge location 
where the angle has passed through a local minima; and 0, denotes a shadowed range bin. Using this synthesized 
measurement, edge features can be detected and classified by the identical method used for the actual WxR measurements in 
the WxR-derived thread. 



Figure 2. Feature-based disparity checking algorithm using forward-looking scanning sensor 



Range bin 


Figure 3. Segment of sample radial measurement and shadow edge features 


At this point in the algorithm, one-to-one disparity checking could be performed since both threads have derived features in 
the same frame of reference. For example, comparison of rising edge or falling edge range bins along each radial could be 
used as the disparity metric. However, due to the nature of the measurements, it is expected that significant missed 
detections and false alarms will occur using this approach. In addition, by only looking at range bins where edges are 
detected, the approach does not take advantage of all the range bin values that do agree in the shadowed regions. For 
example, consider a large shadow region along a given radial. 
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The above observation has led to a disparity checking function that uses “range to center of shadow” as the prototype test 
statistic for disparity checking. 

To provide additional confidence to the estimate of disparity, multiple measurements can be accumulated in each thread and 
shadow size can be monitored with respect to azimuth coverage (i.e. width). This capability completes the initial prototype 
for disparity checking. A shadow size is selected, a priori, and is specified by its coverage, (number of azimuth 
measurements) x (number of range bins). The disparity checker monitors for shadows of this size and compares the center 
locations of these shadows coming from both the WxR-derived thread and the DEM-derived thread. As will be shown, the 
trade-off using this approach becomes choosing a large shadow size that may result in few detections and infrequent disparity 
checks versus choosing a small shadow size that may result in missed detections and false alarms. 

To test the approach presented in this section, flight- testing in the JNU region was performed to acquire a comprehensive set 
of sensor data in an area of severe terrain. 


4. FLIGHT TEST ENVIRONMENT 


A flight test was performed in the vicinity of JNU from September 10 through September 17, 2002. The flight-test aircraft 
was Ohio University’s King Air C90 (Figure 4). 



Figure 4. (a) Ohio University King Air C90, and (b) block diagram of flight- test equipment 
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The experimental system used for this test is shown in Figure 4. Ranging measurements were provided by a commercially 
certified Bendix RS-81 1A WxR and a King KRA405B Radar Altimeter (RAD). Attitude and airspeed measurements were 
provided by an IRU and an Air Data Computer (ADC), respectively. Accurate positioning measurements were provided by a 
Positioning Processor consisting of a Novatel Euro-4 receiver and a Condor Engineering ARINC 429/453 interface card. The 
Positioning Processor was capable of using corrections provided by either the Wide-Area Augmentation System (WAAS) or 
a Local- Area Augmentation System (LAAS) prototype ground station via a data radio. The display processor receives 
aircraft state information from the other processors and combines it with a locally stored DEM to provide a virtual visual 
environment to the pilot on a flat panel liquid crystal display (LCD). This display software was developed by Delft 
University of Technology and is used through a memorandum of agreement. Figure 5 shows the installation of the display on 
the right side of the King Air’s flight deck. The Bendix WxR display is shown just to the left of the experimental LCD in 
Figure 5. 



Figure 5. Installation of Synthetic Vision LCD for co-pilot in the King- Air flight deck. 


The photographs shown in Figure 5 were taken while flying down the Gastineau channel approaching Runway 08. At this 
point in the flight, the aircraft is below the terrain on either side of the flight path. This is indicated by the SVS display as red 
areas on the terrain. More importantly, note how these same areas are black on the WxR display. These shadow effects are 
what we seek to capitalize upon for the prototype integrity monitor algorithm. 

A total of 41 approaches were flown to Runway 08 and Runway 26 at JNU (Figure 6). 
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Figure 6. (a) Approaches flown during flight-testing, and (b) selected flight profiles 
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5. RESULTS 


Using the data collected at JNU, the prototype algorithms have been tested to validate the approach. To illustrate the 
processing steps described in Section 3, consider a representative scan taken on September 12 while flying down the 
Gastineau channel approach Runway 08. The WxR measurements are shown in Figure 7a. For all data collected at JNU, the 
scan range was set to +/- 45 degrees and range was set to 10 nmi. The four color levels indicate increasing reflectivity levels 
detected by the WxR ranging from black (no detectable reflectivity) to blue to yellow to red (maximum reflectivity). Figure 
7b depicts relative terrain heights extracted from the DEM over the same coverage area. Red indicates terrain is higher than 
the aircraft altitude. The other contours are at decreasing levels to black. Black areas of terrain are greater than 250 meters 
below the aircraft altitude. From Figure 7b the Gastineau channel is apparent in the DEM along with several tributaries that 
feed the channel. 



Figure 7. (a) WxR measurements, and (b) DEM contours within WxR field-of-view 


Following the algorithm proposed in Section 3, the first step in each thread is to detect edge features along each radial and 
classify these edges. For the sample scan shown in Figure 7, the results of feature detection and classification are shown in 
Figure 8 for each thread. Visually there appears to be good agreement for this scan. However, as described in Section 3, 
comparing these edge locations directly will not be a good indicator of the actual parity (agreement), as it does not consider 
the agreement of all of the shadow cells between the edges that do agree. 


14 


14 


12 


12 




Figure 8. (a) WxR-derived edge features, and (b) DEM-derived edge features 
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Figure 9 shows the locations of shadow center points for two sizes of shadows and the resulting disparity. These shadow 
center points were computed as described in Section 3. Sizes were chosen to be (13 x 55) and (52 x 110). These sizes 
correspond to (3 deg x 1 nmi) and (12 deg x 2 nmi), respectively. Figures 9a and 9b provide three distinct plots to illustrate 
the resulting effect on disparity checking. The top two plots represent the WxR-measurements (top) and the DEM-derived 
measurements (middle) in an image-space defined by the number of range bins (1 to 512) and the number of radial samples 
(1 to 384). 384 is the number of azimuth radial measurements that are made as the antenna scans from -45 degrees to +45 
degrees and back again. In other words, the x-axis is time and the y-axis is range. The pink pixels are those points in the 
images that have been found to be center points for the shadow sizes specified. When comparing Figure 9a to Figure 9b, 
significantly fewer shadow center points are detected when using a larger shadow size, as expected. 



Figure 9. Measurements, shadow center points, and MSD disparity for shadow sizes 
(a) 3 degrees by 1 nautical miles, and (b) 12 degrees by 2 nautical miles 


The bottom plots in Figure 9 provide an example of a disparity metric that could be used by the integrity monitor. For this 
example, the Mean-Square Difference (MSD) is computed to the first shadow center point along each radial. MSD is 
computed using the following equation: 


MSD = 


( 1 \ N 


c demUY) 


where N is the number of samples and c is the center point range bin for the first shadow found along radial i by the WxR- 
derived thread and the DEM-derived thread. The number of samples used for the MSD is 20 for the plots in Figure 9. Notice 
the effect on the MSD when using a larger shadow size. 

Finally, detection of DEM errors is tested. For the sample scan shown previously, Figure 10 provides plots of the MSD 
disparity metric in the presence of biases inserted into the DEM. The number of samples used to compute the MSD for these 
cases is 50. For Figures 10a and 10b, the bias ranges from 0 (top), to 200 m, to 400 m, to 800 m, and to 1600 m (bottom). 
For Figure 10c, the bias ranges from 0 (top), to 100 m, and to 200 m (bottom). 

Referring to Figure 10, using a detection threshold of 20000, the monitor will detect disparity in the presence of an 800-meter 
along-track bias in approximately 180 samples. At a measurement rate of 6 msec, this corresponds to about 1.08 seconds. 
The monitor will detect a 100-meter cross-track bias in approximately 200 samples (1.2 seconds), and a 200-meter vertical 
bias in approximately 140 samples (0.84 seconds). In addition, it is apparent that as the bias becomes larger, it will be 
detected sooner. 

Figure 10 also indicates the type of terrain we are flying over. In this particular scan while flying along the channel, edges 
are primarily vertical, or along-track. This makes disparity detection much more sensitive to cross-track errors, while along- 
track errors are more difficult to detect. 
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Figure 10. MSD disparity metric in the presence of DEM bias (a) along-track, (b) cross-track, and (c) vertical 


6. DISCUSSION 

Preliminary results indicate that the proposed algorithm has the potential to enable the use of commercial-of-the-shelf 
(COTS) X-band WxR systems for real-time DEM integrity monitoring; however, several issues remain that will be addressed 
as the research continues. These include (1) performing operational assessments, and (2) determining optimum detection and 
disparity metrics. 

6.1 Operational assessments 

Operational assessments are important in order to fully characterize the flight environment with respect to the features that 
may be used for disparity checking. The current algorithm assumes shadowing effects will be observable in WxR 
measurements. This will be a function of terrain, WxR antenna pointing direction, and proximity to the terrain. From the 
JNU data, flight segments at the higher altitudes do not provide results as good as the segments at the lower altitudes. This is 
primarily due to the loss of angular resolution at the longer ranges, but also due to the fact that at altitude, it is less likely that 
the WxR antenna beam pattern is illuminating terrain. Fortunately, phases of flight requiring high integrity for SVS DEMs 
will be when the aircraft is close to terrain (e.g. approach and departure phases). By triggering the integrity monitor 
operation at a prescribed altitude above the ground (e.g. 2000 feet), the concern becomes alleviated. The use of radar 
altimeter measurements to provide this triggering seems appropriate. 

A second issue to address during operational assessment is real-time feasibility. A real-time implementation of the proposed 
algorithm is being developed and will be tested during the summer of 2003 in the Nevada area using NASA’s B-757 test 
aircraft equipped with the Rockwell-Collins WxR-2100. The most significant real-time challenge will be accessing the DEM 
and generating the DEM-derived radial measurements every 6 msec for comparison with the WxR radial measurements. One 
mitigation strategy is to perform less frequent comparison of features at the expense of longer detection times. 

Lastly, quantifying nominal sensor errors in the operational environment is needed to establish minimum detectable DEM 
errors and to determine appropriate disparity thresholds for the integrity monitor. 

6.2 Optimal detection and disparity metrics 

Several options are being considered to optimize the feature detection method. Initially, it is proposed to choose a shadow 
size prior to flight and derive a disparity test statistic and alert thresholds based on this size. An alternative approach would 
be to use a dynamic method. Using this approach, shadow size may be used as the disparity metric. 

With fixed shadow sizes, there is a distinct trade-off between small shadows and large shadows. As shadow size increases, 
there will be fewer, if any, center points to use for disparity checking; however, for the points that are found, the likelihood of 
agreement will increase. The disadvantages of using large shadows are (1) small errors will not be as detectable, and (2) as 
shadow size increases, the likelihood of the WxR observing one in a particular scan will decrease. Thereby, reducing the 
availability of the monitor. 
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On the other hand, using small shadows will detect small disparities (e.g. a peak that is missing in the DEM), but will also 
provide more false alarms due to nominal noise-like errors in the WxR measurements and the DEM. One behavior that is 
common in the JNU data when using small shadow sizes is a detection by one thread and not by the other. Disparity 
checking in this case will result in a large difference that may lead to a false alert. To mitigate this behavior, repeated scans 
can be accumulated to see if the feature persists in one thread and not the other. If it does, this indicates an actual error that 
should be detected. Using multiple scans in this way will improve integrity but must be traded against the increase in time- 
to-alarm that results from waiting for additional scan measurements. The other benefit of using small shadows is that as 
shadow size gets smaller, the likelihood of observing shadows will increase. This increases the availability of the integrity 
monitor. 

Finally, the choice of disparity metric must be made using a statistical approach. This allows the resulting integrity to be 
quantified and/or bounded as is often required for certification. The use of MSD has been shown to be effective for radar 
altimeter implementations of a DEM integrity monitor. Unfortunately, the forward-looking sensor approach suggested in this 
paper must consider nominal sensor errors that were not included in the radar altimeter statistical derivation of disparity test 
statistics and appropriate thresholds. These include attitude errors coming from the IRU, antenna pointing direction errors, 
and the ranging sensor signal-in-space characteristics. The nominal effect of all of these errors on the features extracted from 
both threads must be well understood in the fault-free case in order to set appropriate thresholds to indicate the detection of 
off-nominal, or faulty, cases. 


7. SUMMARY 

An approach to real-time DEM integrity monitoring using a forward-looking sensor has been presented. This approach 
requires a sensor that makes ranging measurements during scanning. Also required is a positioning system and an attitude 
and heading reference system. Using these systems, edge features are detected and extracted from the scanning ranging 
sensor. DEM-derived edge features are also detected and extracted in a parallel thread. Disparity checking is performed over 
areas between the detected edges. An example implementation using X-band Weather Radar is presented that uses shadow 
areas (i.e. regions between edges with zero reflectivity) for disparity checking. The prototype algorithm has been tested 
against data acquired in Juneau, Alaska, and the results suggest the approach is valid, but additional work is required to 
establish optimum performance and to verify real-time feasibility. 
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